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ABSTRACT 

A Study compared the effects on student achievement 
and retention between the problem-solving approach and the 
subject-matter approach in teaching a selected problem area in 
vocational agriculture. A quasi-experimental design, a variation of a 
nonequinalent control group design, was used. Treatments were 
randomly assigned to classes enrolled in introductory vocational 
agriculture course*; in Illinois high schools, resulting in 66 
studonts in the problem-solving treatment group and 60 in the 
subject-matter treatment group. Findings indicated that the 
problem-solving approach was no more or less effective as measured by 
student achievement, regardless of ^he cognitive level of the 
questions. It was no more or less etfective in producing higher 
scores on the delayed retention test, regardless of the cognitive 
level of the questions. For tho high-level cognitive items, students 
taught by the problem-solving approach exhibit lower achievement 
loss. Students' attitudes toward the problem-solving approach were 
slightly more positive, but the difference was not great enough to be 
considered ^statistically significant. (YLB) 
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Throughou! the history of agricultural education, the problem solving approach has beeii 
reconimended by agricultural teacher educators as the primary strategy for teaching. Several authors 
of agricultural education texts cite the problem solving approach as the most effective approach for 
teaching vocational agriculture (Binl^ley & Tulloch, 1981; Crunkilton & Krebs, 1982; Newcomb, 
McCrad^en, & Warmbrod, 1986; F.ipps, 1980). 



Some agricultural educators have questioned the effectiveness of problem solving instruction, 
claiming that changes in vocational agriculture programs and in student backgrounds resulting from 
the expansion of vocational agriculture programs beyond agricultural production have reduced the 
effectiveness of the problem solving approach (Moore & Moore, 1984). Others have argued that 
problem solving instruction results in improved learning by students (Warmbrod, 1969) and 
increased retention of knowledge (Binkley & Tulk)Ch, 1981; Bruner, 1961). 

Little empirical evidence exists to support the problem solving approach to teaching vocational 
agriculture. Studies conducted by Thompson and Tom (1957) and Dawson (1956), which compared the 
problem solving approach with teacher-centered instructional approaches, found some advantages in 
increased student achievement in favor of problem solving instruction, but were conducted prior to 
the expansion of vocational agriculiure programs resulting from the 1963 Vocational Education Act. If 
agricultural educators are to continue to promote problem solving as the most effective approach to 
teaching vocational agriculture, there is a need to examine empirically the effectivenesr of the 
problem solving approach in today's vocational agriculture classroom in terms of two of the most 
commonly accepted measures of teaching effectiveness - student achievement and retention of 
knowledge. 



Purpose 



The purpose of this study was to compare the ejects on student achievement and retention 
between the problem solving approach and the subject matter approach in teaching a selected problem 
area in vocational agriculture. The following research hypotheses provided the framework for this 
study: 
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1. Studer^': taught by the problem solving approach will earn higher scores on a problem area 
achievement test administered at the conclusion of the problem area than student? taught by the 
subject matter apprrach. 

2. The problem solving approach will result in higher student achievement scores for low level 
cognitive items and high level cognitive items than the subject natter approach. 

3. Students taught by the problem solving approach will earn higher s jores on a problem area 
retention test administered one week after the ?chieverrjent test than students taught by the subject 
matter approach. 

4. The problem "x)lving approach will result in higher student retention of information for low 
level a-id h'gh level cognitive items on the problem area retention iGst than the odbjpct matter 
approach. o...^^?!r^.7S^:^!.^'^^^^^ "PERMISSION TO REPPODUCE th:s 
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5. Students taught by the problem solving approach will exhibit less achievement loss as 
rneasured by differences in scores for the .achievement test and retention test, than students' taught by 
tpa subject matter approach. 

6. Students taught by the problem solving approach will exhibit more favorable attitudes toward 
the method of instruction than students taught by the subject matter approach. 

Procedur 

Population and Sampte 

The population consisted of high school students enrolled in introductory vccation a' agriculture 
courses in Illinois. A cluster sampling technique was used to select a purposive sample from the 
target population. Only teachers who taught two or more introductory vocational agriculture classes 
and had prior knowledge and experience in using the problem solving approach were considered as 
possible participants in this study. In order to control for overall teaching ability, the teachers 
taught one class i :ing the problem solving approach and one dass using the subject matter approach. 

Since this study was conducted using intact groups, random assignment of students to treatments 
was not possible. A quasi-experimental design, a va iation of nonequivalent control group design, was 
used for this study. Using a formula suggested by Hays (1973), which incorporated the desired alpha 
level, power of the and effect size, a necessary sample size of 60 students in each group was 
determined as appropriate. This sample size was based upon an alpha level of .05, a desired powe. of 
.90. and an effect size of .66 standard deviations. 

Telephone interviews were conducted with 20 teachers who taught two or more introductory 
vocational agriculture classes to determine their knowledge and use of the problerr solving approach 
A lis of steps involved in using the problem solving approach was used to determine if teachers we. e 
familiar with all of the steps in using problem solving instruction. Four teachers who indicated 
dunng the interviews they used all of the steps in the problem solving approach were selected and 
agreed to participate. Treatments were randomly assigned to the classes, resulting In 68 students in 
the problem solving treatment group ard 61 students in the subject matter treatment group Due to 
absences during the testing period, data were collected from 66 students in the problem solving 
treatment group and 60 students in the subject matter treatment group. 

Instrumentatinn 

An instrumen > measure student attitudes toward the teachino approaches was developed by the 
researcher. Instruments developed by the researchers to measure the achievement and retention 
variables consisted of a problem area achievement test and a parallel problem srea retention test 
Mehrens and Lehman (1973) described parallel tests as those with equal content, constructed using 
the same sper;<ications, of similar difficulty, and using the same format. Both instruments were 
constructed to include both high level and low level cognitive items. Content validity of the 
instruments was established by a panel of experts consisting of two members of the agricultural 
education faculty at the University of Illinois and two high school vocational agriculture teachers The 
panel also evaluated the difTicuity level of the item^ on the instruments and determined that the tests 
were equal in content. In addition, student scores for both instruments in the pilot test were not 
significantly different. The university faculty members rated each item as either a low level or a high 
level cognitive Item using the cognitive levels described by Bloom (1956). For this study low level 
cognitive items were those which required knowledge and comprehension, and high level cognitive 
Items required application, analysis, synthesis, or evaluation. 
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The instruments were field tested and pilot tested for darity and reliability using students in 
two vocational agriculture departments not participating in the study. The Kuder-Richardson 20 
reliability coefficient for the revised student achievement instrument was .80, and a K-R 20 
coefficient of .88 was calculated for the retention instrument. Heliability coefficients for the low 
level cognitive scale and high level cognitive scale on the achievemen< and retention instruments 
ranged from .73 to .91 . Cronbach's alpha coefficient of internal consistency was .95 for the 
instrument measuring student attitude toward the teaching method. 

Data Collection 

Prior to the beginning of the study, teachers selected to participate received inservice education 
on the proper use of the two teaching approaches. The problem solving approach was a 
student-centered instructional approach, while the subject matter approach in this study was a 
teacher-centered approach. While the problem solving approach focused instmction on problems 
and concerns raised by the students, the subject matter approach focused upon agricultural subject 
matter selected by the teacher. With the problem solving approach emphasis was placed upon using 
information to develop solutions to problems, while the subject matter approach emphasized learning 
agncultural facts. Specific teaching techniques and learning activities to be used during the problem 
solution step (problem solving approach) and the presentation step (subject matter approach) were 
determined by the teachers. 

The steps involved in the problem solving approach, as outlined by Crunkilton and Krebs 
(1982), included conducting an interest approach, developing student objectives for studying the 
problem area, laentification r^f specific problems and concerns of students related to the prcblem area, 
developing possible solutio' is to problems and concerns, orawing conclusions or summarizing the 
appropriate iolutions, and student evaluation. The steps involved in the subject matter approach 
included an introduction (in which the teacher presented the reasons for studving the problem area), 
presentation of subject matter, review of important points, and student eva'uation. 

Data were collected in all four schools between mid-April and early May of 1986. A problem 
area in com production which required approximately 10 dass periods of instmctional time was 
selected for this study, since com production was included in the course of study for each of the 
schools £ -ilected. The problem area focused upon seedbed preoaration and planting procedure : for corn. 
Each claswi participatinc in the study was visited by tht resea/cher during the study to observe the 
teaching approaches used and to verify that the proper ireatmeits were being used in the assigned 
groups. The problem area achievement test and attitude toward the teaching method instrument were 
administered immediately following the treatment, and the problem area retention test was 
administered one week following the achievement test. In order to control for possible preexisting 
group differences, average grades for the first semester of vocational agriculture and student IQ scores 
were collected for each student to be used as *'X)variate measures in the study. 

Analysis of nata 

For the purpose of statistical analysis, appropriate null hypotheses were developed for each 
research hypothesis and were tested at the .05 alpha level. Multivariate analysis of covariance 
(MANCOVA) techniques were used to examine the data, which allowed the two dependent variables to be 
examined simultaneously and took into account existing correlations between the variables. 
Hotelling's t2 statistic was used to determine if differences existed between the two treatment groups 
with univariate analysis of covariance used as a follow-up procedure. The selection of covariate 
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measures was supported by within group correlations computed for the covariate measures and 
Ltudent achievement and retention scores which indicated that the covariate measures were related to 
the major dependent variables, in addition, t-tests indicated significant differences between treatment 
groups for both average grade in vocational agriculture and student IQ, suggesting that analysis of 
covariance techn'ques were appropriate. Student attitude toward the teaching method was not assumed 
to be con'ela.ed with the covariate measures. Therefore, the student attitude variable was not included 
In the multivariate analysis of dependent variables. A t-test was used to determine if significant 
di^erences ii student attitude toward the teaching method existed between the Veatment groups. The 
data were al:^o described using measures of central tendency, measures of variance, frequencies, and 
percentages. 

Results 

A multivariate analysis of covariance was performed on student scores for the problem area 
achievement test and the problem area retention test. Hotelling's statistic was .041 [F (2, 121) = 
2.50, p».08], indicating there was no difference at the .05 alpha level between the treatment groups 
when student achievement and student retention were considered simultaneously. In addition, 
univariate follow up tests indicated no difference in student achievement or retention betwee.i the 
groups when cognitive levels of the iiems were considered. Therefore, the data did not support 
research hypotheses one through four. 

Findings Related to .qturipnt Achieveinent 

Student achievement was measured by the number of con-ect responses on the 25-item problem 
area achievement test. Due :c pre-treatment differences between the groups, the mean scores for 
student achievement were adjusted using the covariate measures. Summary statistics for student 
performance on the problem area achievement test which included 16 low level cognitive items and 9 
high level cognitive items are presented in Table I. 



Table 1 

Mean Student Achievement for Problem Area Achiftve ment Test bv Teachin? Method 







Total score 


Low level items 


High level items 


Teaching method 


n 


Observed x Adjusted x 


Observed x Adjusted x 


Observed x Adjusted ^ 


Problem solving 


66 


13.59 13.13 


9.55 9.20 


4.05 3.92 


Subject matter 


60 


13.38 13.85 


9.10 9.44 


4 28 4.40 



Findings Related to RPtPnt ion of Knowledge 

Two measures of retention of acquired knowledge were used: (a) the number of con-ect 
responses on the 25-item problem area retention test and (b) achievement loss, or the difference 
between student scores on the achievement test and the parallel retention test. Student scores on the 
retention test were adjusted using the covariate measures of average grade in vocational agriculture 
and iQ scores. Summary statistics for student performance on the retention test are presented in 
Table 2. 
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Table 2 

Mean Student Retention for Problem Area Ratention Test bv Teachiny Methods 







Total score 


Low level items 


High level items 


Teaching method 


D 


Observed x Adjusted x 


Observed x Adjusted x 


Observed x Adjusted x 


Problem solving 


66 


12.24 11.89 


8.02 7.75 


4.23 4.14 


Subject matter 


60 


10.90 11.25 


7.27 7.53 


3.63 3.73 



As shown in Tables I and 2, students in the subject matter treatment group had slightly higher 
adjusted mean scores on the achievement test than students in the problem solving treatment group. 
However, the reverse was found for the retention test, with students in the problem solving group 
earning higher adjusted mean scores than studen'is in the subject matter treainierit group. VVnlle lu^ 
differences between tho groups were not significant for either achievement or retention, this reversal 
indicated a lower achievement loss for students in the problem solving treatment group (see Table 3) 



Table 3 

Mean Student Achieveme nt Loss by Teaching Method 







Total achievement 


Achievement loss 


Ach'evement loss 


Teaching method 


D 


loss 


for low level items 


for high level items 


Problem sotvltig 


66 


-1.24 


-1.45 


+0.21 


Subject matter 


60 


-2.60 


-1.91 


-0.67 



USi^. Mean student achievement loss was calculated from mean scopes adjusted for covariate measures. 
Negative values indicate achievement loss. Positive values indicate achievement gam. 



Because of this reversal in scores, an exploratory analysis of variance was performed for 
achievement loss on the adjusted mean scores for the two groups to determine if the difference in 
achievement loss was statistically significant (see Table 4). When the total test scores were 
considered, there was a significant difference in achievement loss in favor of the problem solving 
treatment group. Research hypothesis five was supposed by the data, and the hypothesis of no 
difference in achievement loss between students trughi oy the problem solving approach and students 
taught by the subject matter approach was rejected. 
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Table 4 

ANCQVA Summary Table for Afthievemftnt Loss 



Source of variation 


SS 


df 


MS 


F 


Covarlates 


13.33 


2 


6.67 


0.54 


Average grade 


5.62 


1 


5.62 


0.46 


IQ 


1.11 


1 


1.11 


0.09 


Teaching method 


54.60 


1 


54.60 


4.40* 


Residual (error) 


1500.52 


122 


12.30 




Total 


1568.44 


125 







•p < .05. 



In order to examine the nature of the difference in achievement loss between the two treatment 
groups, further analysio was performed on achievement loss for the two cognitive levels of the items. 
For low level cognitive items, achievement loss for the problem solving group was slightly lower, but 
not significantly lower, than for the subject matter group [F (1,122) 0.84, p = .36]. The 
sub-hypothesis of no difference in achievement loss for low level cognitive items was not rejected. 

However, for high level cognitive Hems, students in the problem solving treatment group scored 
slightly higher on the retention test, resulting in an achievement gain, while students in the subject 
matter group experienced an achievement loss. An analysis of covariance was performed to determine 
the significance of the difference between the treatment groups. The F test indicated a significant 
difference between the treatment groups (see Table 5), indicating that for high level cognitive items, 
students taught by the problem solving approach had less achievement loss than students taught by the 
subject matter approach. Therefore, the data supported the sub-hypothesis associated with research 
hypothesis five, and the hypothesis of no difference between treatment groups for achievement loss for 
high level cognitive items was rejected. 



Table 5 

ANCQVA Summary TabiP for Achievfimftnt Loss for Hioh Level Items 



Source of variatiori 


SS 


df 


MS 


F 


Cova tes 


5.69 


2 


2.85 


0.82 


Average grade 


3.82 


1 


3.82 


1.10 


IQ 


4.88 


1 


4.88 


1.40 


Teaching method 


23.59 


1 


23.59 


6.79* 


Residual (error) 


423.94 


122 


12.30 




Total 


453.21 


125 







•p < .01 . 
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Findings Related to Student Attitude Toward the Te aching Method 

Student attitude toward the teaching methods examined in this study was measured by student 
responses on the 24-item Attitude Toward Teaching Method instrument developed by the researchers. 
Students were asked to respond to each item on a Ukert-type scale which ranged from Strongly Agree 
to Strongly Disagree. Responses were assigned weights ranging from one to five points. Negatively 
stated items were reverse scored. The maximum possible score on the attitude instrument was 120. 
Student scores on the attitude instrument ranged from 32 to 1 12. Summary statistics for student 
attitudes toward the teaching methods are presented in Table 6. 



Table 6 

M fean Scores for the Attitude Toward Teaching Met hod bv Teaching Method 



Teaching method 


n 


X 


SD 


Problem solving 


66 


77.14 


15.86 


Subject matter 


60 


72.45 


17.29 



A t was used to test the hypothesis of no difference in student attitudes toward the teaching 
approaches examined in this study. There was no significant difference in student attitude between 
students taught by the problem solving approach and students taught by the subject matter approach 
l'{124) ^-59. P * •12]. Therefore, the hypothesis of no difference was not rejected. 



Conclusions 

The conclusions drawn from this study were limited to the extent that the sample was not 
randomly selected from the population. Therefore, the conclusions are generalizable to the extent that 
the sample was representative of the population of students enrolled in introductory vocational 
agriculture courses in Illinois. Bas^d upon the findings of this study, the following conclusions were 
drawn: 

1. The problem solving approach is no more or less effective than the subject matter approach as 
measured by student achievement, regardless of the cognitive level of the questions. 

2. The problem solving approach is no more or less effective than the subject matter approach in 
producing higher scores on the delayed retention tesi, regardless of the cognitive level of the questions. 

3. For high level cognitive items, students taught by the problem solving approach exhibit lower 
achievement loss than students taught by the subject matter approacn. 

4. Students' attitudes toward the problem solving approach are slightly more positive than 
students' attitudes toward the subject matter approach, but the difference is not great enough to be 
considered statistically significant. 
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Recommendations 

1. Because of the slight advantage of the problem solving approach in the? area of student retention 
of knowledge, the problem solving approach may be used with confidence to teach introductory 
vocational agriculture course^. 

2. Studies of similar purpose and design with a variety of problem areas and students from both 
rural and urban backgrounds should be conducted in vocational agriculture classrooms to increase the 
generalizabillty of the findings of this study. 

3. Additional studies should be conducted to determine the effect of the problem solving approach 
on retentk)n over longer periods of time. 

4. Studies involving students in vocational agriculture courses should bo conducted to determine 
the effectivoness of problem solving instruction in improving problem solving and decision making 
skills. 
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